Implementation of a Peer Review Process to Improve Documentation Consistency of Care Process Indicators in the EMR in a Primary Care Setting

BACKGROUND: Kaiser Permanente Colorado is a group model nonprofit HMO that provides health care services to more than 500,000 members. The Primary Care Clinical Pharmacy Services (PCCPS) department consists of 33 clinical pharmacy specialists (CPS), who are located in 19 primary care clinics. OBJECTIVES: To develop and implement a peer review process to (a) improve the consistency of documentation of process indicators in the electronic medical record (EMR), (b) ensure compliance with existing standards, and (c) share best practices among PCCPS with varying geographical locations and practice styles. METHODS: A committee was formed to undertake the peer review process. An audit tool consisting of yes/no questions was created to assess chart documentation by PCCPS and to provide feedback for improvement. Four sections were included in the evaluation tool: (a) content, (b) collaborative drug therapy management, (c) nonformulary reviews, and (d) pharmacy system documentation. Peer reviews occurred quarterly, and all CPS participated. Copies of reviews were distributed to PCCPS clinicians and their supervisors. Questions and inconsistencies regarding the process were identified by the peer review committee to provide feedback to the group to optimize reviews. After completion of each quarter’s reviews, error rates were calculated by dividing the total number of “no” answers by the total number of PCCPS notes reviewed that quarter. A 2-tailed Fisher’s exact test was used to compare the error rate at the last quarter of each year (2007 to 2010) with baseline (2007 Q1). RESULTS: A total of 1,856 reviews were conducted between 2007 Q1 and 2010 Q2. Significant improvements in documentation were demonstrated over the first 12 months and sustained for the next 2.5 years. From 2007 Q1 to 2010 Q2, the rate of noncompliant elements decreased from 14.1% to 2.5% (P = 0.001) in the content section and decreased from 31.3% to 8.3% (P  less than  0.001) across all sections. CONCLUSIONS: Over 3 years of follow-up, the peer review process was successful in improving the consistency of documentation by PCCPS and compliance with existing standards. The process was well received by participants. The peer review document is easily adaptable and can be updated to address changes in drug therapy management protocols and nonformulary medication reviews as needed. This process also allows for sharing of best practices among high-functioning PCCPS practitioners who otherwise could remain isolated.

• Use of peer review for the practice of pharmacy has not been well described in the literature. Published reports in the inpatient and outpatient pharmacy settings have demonstrated that the peer review process has been successful in improving the documentation by pharmacists in the medical record, has been viewed as constructive, and has been well supported by participants. When rated by a committee of 6 nonsupervisory clinical pharmacists, pharmacist care was rated as level 1 (most "competent practitioners would have handled the case similarly") in 236 of 250 cases (94.4%) and as level 2 (most "might have handled the case differently") in 14 of 250 cases (5.6%). No cases were rated as level 3 (all "would have handled the case differently"). Process indicators, such as documentation of medication reconciliation and patient adherence, improved over time, and clinical pharmacists surveyed indicated support for the peer review process and felt it should be continued. • Cram et al. (1992) used a peer review system for monitoring pharmacist performance in medication refill clinics in a VA setting. Quarterly audits of approximately 10% of charts from the previous 3 months were conducted by pharmacists working in the medication-refill clinics to ensure compliance with 5 quality indicators, including documentation format and detail, appropriateness of recommendations, and compliance with policies and procedures. Results from 4 quarters of peer reviews did not demonstrate significant increases in the 5 indicators; however, improvements were noted in documentation of medication lists for all patients and for dating and ordering laboratory work. • In an inpatient setting, Zimmerman et al. (1997) described continuous quality improvement activities by a peer review group of pharmacists. Four key indicators were used for reviews: (a) clinical appropriateness of interventions, (b) accuracy of computer entry of interventions, (c) pharmacist documentation of the intervention in the medical record, and (d) implementation of accepted interventions. A total of 409 interventions were evaluated over 9 meetings: 96% of the interventions were rated clinically appropriate; 62% were entered into the computer accurately; 62% were documented; and 92% were implemented if accepted by the prescriber.

What is already known about this subject
Implementation of a Peer Review Process to Improve Documentation Consistency of Care Process Indicators in the EMR in a Primary Care Setting article is to describe the development and implementation of a peer review process at Kaiser Permanente Colorado (KPCO) and report the resulting changes in documentation measured from 2007 to 2010.
KPCO is a group model nonprofit health maintenance organization that provides health care services to more than 500,000 members. The Primary Care Clinical Pharmacy Services (PCCPS) department consists of 33 clinical pharmacy specialists (CPS) in 19 primary care clinics located throughout the front range of Colorado. Clinics are separated by distances of up to 50 miles. The CPS clinicians have doctor of pharmacy degrees and residency training or equivalent experience and are expected to become board certified as pharmacotherapy specialists within 3 years of employment.
Due to the independent nature of the practice setting and the variety of activities in which CPS are involved within their respective clinics, documentation and practice styles vary significantly. There is wide variation in the way the CPS support the members of the health care team and the patient population at each clinic. Provider needs, CPS interest areas, or the diversity of the clinic itself influence the practice and work flow of the individual CPS. Some examples of the types of activities in which CPS are involved in the clinic include recommending and implementing drug therapy, longitudinal follow-up of patients with chronic pain; hypertension, diabetes, or hyperlipidemia; and general education of patients regarding medications, disease states, and adverse drug events.
CPS are responsible for upholding standards for compliance with documentation in the electronic medical record (EMR) within KPCO, especially for certain activities that are regulated by external agencies. For example, PCCPS clinicians should follow collaborative drug therapy management (CDTM) protocols, which are physician-pharmacist agreements that allow CPS to engage in drug therapy management within KPCO. CDTM is regulated by the state of Colorado, and practice standards were developed internally for use within KPCO. Documentation standards specifically address initiation and management of drug therapy for hypertension, diabetes, hyperlipidemia, and anticoagulation. PCCPS members also participate in the review of nonformulary medications by providing recommendations to department chiefs regarding approval or denial. A standardized peer review process was implemented to minimize variation of the documentation within PCCPS, ensure that individuals comply with documentation requirements, and promote a high standard of note quality and sharing of best practices.

■■ Methods
In order to help ensure success of the PCCPS peer review process at KPCO, implementation of the process was loosely modeled after the 5 steps described by Norcini (2003). 2 To accomplish the stated goals, the PCCPS team formed a Peer P eer review has been described as the continuous, systematic, and critical evaluation of a specific aspect of a practitioner's performance by professional colleagues. 1,2 The peer review process is frequently used to help assess and critique clinical performance with the goal of sharing best practices and increasing the quality of documentation in the medical record. Peer review has been used in the inpatient and outpatient pharmacy settings for the evaluation and monitoring of clinical performance. [3][4][5] The steps in the implementation of a peer assessment of performance for physicians have been described by Norcini (2003). 2 First, state the purpose of the assessment. All participants should be made aware of the purpose of the assessment as well as the expectations of those reviewing and those being reviewed. Second, develop assessment criteria and communicate them to participants, including the details of the review such as who will participate, when reviews will take place, and what is considered acceptable performance. Third, provide training to the participants, ranging from basic written instructions to detailed video-based benchmarking with feedback. Fourth, monitor results of the assessments throughout the process and obtain feedback from participants in order to determine the need for more training or change in data collection. Ongoing checks of the reliability and validity of the process are also recommended. Fifth, provide feedback to participants with reviewers (evaluators) compared with each other in stringency/leniency in order to ensure consistency, and persons reviewed should receive the results of the peer assessment. 2 Haines et al. (2010) described the systematic implementation of a peer review process for staff clinical pharmacists with advanced scopes of practice at a Veterans Affairs (VA) medical center. 6 Improvements in process indicators, such as documentation of medication reconciliation and patient adherence, were observed over the 10-month measurement period. However, no published reports have measured the long-term effects of a peer review process on performance. The purpose of this

What this study adds
Review Committee (PRC) to develop, implement, and evaluate the review process. The initial PRC formed in late 2005 and comprised 4 volunteers from the PCCPS team and 1 PCCPS supervisor. Their initial goals were to create a standardized form to use for peer review, develop the peer review process, and gain support from the PCCPS team. After performing the initial review for 2007 Q1, it was determined that the magnitude of this task was too large for a committee of 5 to maintain. This conclusion, along with an increased interest in the committee from PCCPS, resulted in expansion of the PRC membership. It has since fluctuated between 5 and 12 members and is currently composed of 10 persons, including the chairperson and the PCCPS supervisor liaison. Length of membership is not currently limited but typically lasts for 2 to 3 years in order to ensure that new PCCPS staff members have a chance to serve. Members report their participation on the PRC to be extremely beneficial as it helps them familiarize themselves with CDTM and nonformulary requirements and allows them to learn from senior staff members. The chairperson is a current serving member of the PRC and is elected by the PRC. The supervisor liaison does not serve as the PRC chairperson.

Initial Development of Audit Tool
The PRC created a standardized form to be used to review EMR documentation by CPS. Items chosen for inclusion in the audit tool were elements from CDTM and nonformulary requirements that were considered to be essential to compliance with these standards, such as meeting the timeline for a nonformulary medication review. Other items were chosen to ensure compliance with a minimum standard of documentation in the EMR, such as clearly documenting in the subjection section of the note the reason for referral. Such items were included in the content section or scope of practice section of the tool. Lastly, in order to ensure consistent tracking of pharmacy interventions by CPS, questions pertaining to the use of the pharmacy documentation system ("PharmDoc") were included.
All questions on the form are answered initially with "yes" or "no," with a few of the questions allowing for a not applicable (N/A) response. A "no" response indicates failure to comply with the requirement. Examples of the correct use of "N/A" were discussed with CPS for those items that might have some discrepancy of interpretation, such as item 1f, which asks whether a note has physician approval ( Figure 1). For example, if the primary reason for an encounter was strictly to educate the patient and no change in drug therapy was recommended, then an "N/A" would be appropriate because the provider's signature was not necessary.
A comments section was incorporated after each question so the reviewer could include subjective comments on positive aspects of the notes and offer recommendations for improve-ment or alternative ideas for the therapeutic plan. Each item was developed to minimize subjective personal opinion feedback and to maximize the objectivity of the feedback to ensure that all of PCCPS was documenting to a minimum standard of care while promoting improvement in documentation and sharing of best practices.

Implementation and Modification of Audit Tool
The audit tool, expectations for review, and a timeline for the process were presented to the entire PCCPS team in 2006 for discussion at a monthly staff meeting. The tool was then modified based on group feedback. Since the onset of peer review, the data collection form has been updated a total of 4 times in 2007 Q3, 2007 Q4, 2009 Q2, and 2010 Q2. The data collection form originally comprised 5 sections, each containing up to 9 questions in a given area for a total of 28 questions. In response to feelings of survey burden expressed by the PCCPS team, the data collection form was streamlined in 2009 Q2 to include a total of 4 sections: (a) content, (b) CDTM, (c) nonformulary reviews, and (d) PharmDoc documentation. A "scope of practice" section was eliminated, since the majority of questions in this section duplicated those found in the "content" section. Most recently in 2010 Q2, 2 separate questions in the CDTM section regarding laboratory orders and medication changes were condensed into 1 question to be consistent with the format of a similar question in the content section. The current form ( Figure 1) includes a total of 22 questions-18 pertaining to content and 4 pertaining to pharmacy documentation.
Sections 1 (content) and 4 (PharmDoc documentation) are completed for each note, while sections 2 (CDTM) and 3 (nonformulary reviews) are completed as needed, only when a CDTM or a nonformulary note is reviewed. The original forms were completed on paper but are now available electronically and are completed and securely e-mailed by each reviewer to the staff member being reviewed and his or her supervisor. The data collection form can be easily adapted to meet the needs of the PCCPS group and can be quickly revised as standards change or as new areas of focus are identified. Electronic copies of all completed peer reviews are saved by the PCCPS chief for a period of 3 years in case of an audit to demonstrate compliance with internal and external policies.

Responses to Staff Concerns
Initial concerns regarding the process included that it would be used for disciplinary action, that it would limit the individual clinician's practice style, and that it would allow others to openly criticize team members in an unproductive way. These concerns were addressed during group discussion at a PCCPS staff meeting. The PRC provided the following assurances to CPS: a. Rev iews would be used to record an indiv idual 's longitudinal performance and evaluate compliance with  1a. Is the source of the referral or rationale for being in the patient's chart clearly stated?
Are the disease state(s) to be addressed or reason for the referral clearly stated?
Is there a verbal or written order documented for any new, stopped or adjusted medication orders?
Are the medications and labs ordered accurately?
If appropriate or necessary, is follow-up addressed? CDTM, nonformulary, and content documentation requirements. They would not be grounds for disciplinary action unless the practitioner consistently made the same error throughout without evidence of any efforts to improve. b. The focus of the review was to ensure that necessary components of the note were included. The PRC would avoid excess commentary regarding documentation style. In other words, the review would focus on content of the note rather than attributes such as excess wording, poor grammar, or including repetitive information already noted in the EMR. Different authors might have varying ways of conveying care in a note; however, critical elements should still be consistent between authors. c. Throughout the review process, the PRC would be blinded to the author of each note. d. The PRC would discuss elements of each note and not the author of the note. e. For the initial round of reviews, the author of the note would have the opportunity to review any "no" ratings and comment or explain before the note was sent to his or her supervisor. After the first set of reviews, PCCPS agreed that the peer review process was not threatening and that this step was unnecessary. With subsequent reviews, notes were sent to the author and the author's supervisor simultaneously.

Review Process
To locate specific notes within the EMR to be reviewed, each quarter's retrospective encounter data were collected through PharmDoc, a database separate from the EMR that tracks PCCPS interventions. 7,8 Daily clinical interventions, including referral source, disease state, and action taken, are recorded in PharmDoc by each CPS for every note entered in the EMR. These patient encounters can then be retrieved and downloaded to a Microsoft Excel (Microsoft Corporation, Redmond, WA) spreadsheet. EMR notes were chosen arbitrarily for review from this spreadsheet, but an attempt was made to include those notes involving CDTM or nonformulary reviews because regulatory compliance issues are associated with these types of notes. The initial reviews were completed during 2005 Q4 by members of the PRC. Although inter-rater reliability testing was not performed, 2 arbitrarily selected notes for each CPS were reviewed by 2 separate PRC members for a total of 4 unique notes reviewed for each CPS. Reviewers communicated which specific notes they would be reviewing in order to ensure that duplicate reviews of 1 note did not occur. "No" answers were tallied by section for all notes reviewed, and trends were identified. After each quarterly review but prior to sending out completed reviews, the PRC met as a group to talk about questions related to PCCPS documentation. The group members compared and discussed their findings and judgments on individual reviews and came to a consensus on what was deemed to be acceptable documentation. As a group, they created a list of review topics, which were presented to CPS at the PCCPS staff meeting. In order to clarify discrepancies and reduce variation, the review topics highlighted areas where documentation from the PCCPS department was inconsistent. Examples of both exceptional and poor documentation (each blinded for author) were reviewed during a question-and-answer series for group learning purposes. The discussions at the PCCPS meeting were felt to be valuable for CPS and supervisors. CDTM protocols were revised as needed based on the peer review process because sometimes PCCPS recommendations fell out of established CDTM protocols due to clinically sound interventions. For example, ordering a uric acid level after starting a diuretic was updated in the protocol to apply only to those patients in whom a gout flare might be a concern or if clinically indicated, rather than to all patients indiscriminately in response to compliance audit results.
As the number of PCCPS staff increased between 2005 and 2006, it became impractical for the PRC members to review multiple notes for each PCCPS staff member. Consequently, the PCCPS department agreed to have the entire PCCPS team participate as reviewers of each other's notes. The PCCPS team perceived 2 key benefits for participating in the note review process. First, each PCCPS member would have the chance to see a range of documentation styles and clinical practices. The review process would allow members to learn one another's practice strengths and encourage the sharing of best practices outside of the peer review process. For example, a PCCPS member could learn from a challenging patient case or encounter a template created within the subjective, objective, and plan (SOAP) note to address a certain therapeutic area and incorporate or modify the templates for use in his or her own practice. Second, CPS would receive regular feedback on documentation performance in a planned and structured way.
The PRC trained all PCCPS members on the use of the form prior to their first formal review (2007 Q1), including a review of form elements and a discussion of appropriate constructive feedback. The PCCPS team agreed to maintain focus on objective elements of documentation within a chart note and not on documentation style. Each CPS was assigned a PRC mentor, who was responsible for answering questions during the review process and collecting all notes reviewed from his or her mentees. Questions raised by mentees about documentation or the review process were discussed at the PRC meeting, and the PRC members then provided feedback to their mentees and updated peer reviews as needed prior to returning the reviews to the PCCPS staff member being reviewed. This is an important quality control measure to ensure that colleague feedback is accurate and constructive.
At the end of each quarter's peer reviews, the total number of "no" ratings in each section for all reviewed notes was tallied Implementation of a Peer Review Process to Improve Documentation Consistency of Care Process Indicators in the EMR in a Primary Care Setting of these (942 out of 1,856 reviews) were CDTM notes. It is not known how many nonformulary notes were reviewed because the number of these notes reviewed was not tallied until 2010 Q2. The total number of "no" responses in each section was relatively low overall, indicating a high level of compliance with existing protocols and accepted practice. From baseline to 2010 Q2, there were significant decreases in the rate of "no" responses in the content section and across all sections (content, CDTM, and nonformulary). The rate of noncompliance in the content section decreased from 14.1% (18 of 128 reviews) at baseline to 2.5% (3 of 120) in 2010 Q2 (P = 0.001), and for all sections it decreased from 31.3% (40 of 128) at baseline to 8.3% (10 of 120) in 2010 Q2 (P < 0.001).
Overall results across sections in 2007 Q2 did not differ greatly from those in 2007 Q1 (baseline); however, over the follow-up period, the general trend of noncompliance in all sections except CDTM was downward. Except in the CDTM category, the total "no" rates at the end of each year (Q4) compared with the baseline rate showed consistent significant improvement. In the CDTM section, the pattern of results over the follow-up period was less consistent. There was a dramatic increase in "no" ratings in the CDTM section from 5.0% (5 of 101 reviews) at baseline to 24.6% (16 of 65 reviews) during 2007 Q2, which then decreased to 13.9% (15 of 108 reviews) in 2007 Q3. "No" ratings in the CDTM section continued to fluctuate somewhat from 2007 Q4 to 2010 Q1, ranging from 3.3% to 8.2% during that period. In 2010 Q2, however, there were zero "no" ratings in the CDTM section.
Because the total number of reviews of nonformulary notes was not counted until 2010 Q2, it was not possible to assess the percentage of "no" ratings. However, there was a trend toward improvement from 2007 to 2010 but an increase in the absolute number of "no" ratings in the nonformulary section in 2009 Q1 and Q2, which contributed to a high overall number of "no" ratings in those quarters, 21.0% and 19.0%, respectively. Subsequent improvements in the number of "no" ratings in the nonformulary section were seen in 2009 Q3 and Q4. By 2010 Q2, there was a total of 7 "no" counts for the nonformulary section, compared with 17 at baseline (2007 Q1).  5 demonstrated that initiation of a peer review process resulted in improvement in documentation in the EMR. Additionally, a high percentage of pharmacist interventions was rated highly by peers. In Haines et al., 236 of 250 cases reviewed were rated as a level 1 (most "competent practitioners would have handled the case similarly"). 6 In Zimmerman et al., 96% of the pharmacist interventions were rated clinically appropriate. 3 These studies were performed in the inpatient setting (Zimmerman et al.), an outpatient refill and recorded. The total number of notes and the total number of notes where a CDTM protocol were used were also recorded; however, the total number of nonformulary notes reviewed was not counted until 2010 Q2. The total number of "yes" ratings and the total number of N/A ratings were not tallied. The quarterly rate of "no" ratings, defined as the total number of "no" ratings divided by the total number of notes reviewed each quarter, was calculated. The "no" ratings for the final quarter of each year were compared with baseline (2007 Q1) using a 2-tailed Fisher's exact test to assess for statistical significance. All analyses were performed at an a priori statistical significance level of 0.05. All analyses were done using SAS statistical software (version 9.1.3, Carey, SC).

■■ Results
Results of the peer review process are presented in Table 1.
(Results for PharmDoc are not included in Table 1 because this section is used for internal tracking/cost savings and not for compliance with standards of documentation.) A total of 1,856 reviews were performed between 2007 Q1 and 2010 Q2. All of these notes were reviewed for content. Approximately one-half

Count of "No" Ratings Expressed as Percentage of the Total Number of CPS Notes Reviewed Per Calendar Quarter
been used to point out trends (if any) to practitioners so that they might make necessary changes in their documentation practices. The peer review process is transparent; everyone participates, including the senior management staff, and there is a high level of communication and feedback regarding the process and findings. Due to the flexibility of this process and audit form, it can be adapted to meet the changing needs of the PCCPS team. Furthermore, other groups within KPCO, such as Clinical Pharmacy Specialty Services, have adapted this process and audit tool for use within their services.

Limitations
First, although the flexibility of the KPCO audit tool demonstrates its adaptability to other groups within the organization and conceivably to groups outside of KPCO, it is also one of the main limitations to the results. The frequent revisions of the audit tool, which allowed PCCPS to quickly evaluate group understanding of new processes and procedures, made it difficult to compare findings over time.
Second, the total number of nonformulary notes reviewed was not counted until 2010 Q2, mainly because it was not necessary to track compliance with drug therapy protocols under requirements from the Colorado State Board of Pharmacy. This omission did not allow improvement to be tracked as a percentage of all nonformulary notes reviewed over time. Counts of the absolute number of "no" ratings without a denominator are unreliable because the number of nonformulary notes reviewed affects the count of "no" ratings. With this limitation, the absolute number of "no" ratings in the nonformulary section decreased over time; the total number of nonformulary notes reviewed each quarter is now counted and tracked.
Third, the number of individual notes in which a "no" rating occurred is not known; only the overall total number of "no" ratings. This means that 1 poorly written note, or a single documentation omission, can unduly negatively impact overall compliance because it could contain multiple errors and therefore multiple "no" ratings. For this reason, the overall percentage of "no" ratings might underestimate the group's compliance with standards. Despite this potential limitation, 2010 Q2 data indicate that overall compliance was very high. In 2010 Q2, the percentage of "no" ratings in all sections was 8.3%, down from 31.3% in 2007 Q1. We now collect and track data at the individual note level.
Lastly, inter-rater reliability was not assessed. It would have been useful to conduct this analysis, especially during the initial implementation in order to identify areas of the peer review document that were unclear or areas where reviewers needed more training. The PRC relied instead on discussions of reviewer questions during PRC meetings and verbal feedback from PCCPS regarding the process. No formal evaluations of staff satisfaction or completion time for reviews were clinic (Cram et al.), and a VA health care facility (Haines et al.), resulting in over 9 to 12 months of follow-up with analysis of approximately 250 to 400 reviews in each study. Similar to this work, the PCCPS in the present study demonstrated significant improvement in documentation after initiation of peer review, with almost all notes containing the necessary elements of documentation at study end. Compared with the previous studies, the present study included more than triple the number of reviews and a much longer follow-up period. Significant improvements in EMR documentation were maintained over 3 years of follow-up. This would suggest that a robust peer review process can result in ongoing improvements in documentation, compliance, and sharing of best practices.
Early in 2009, wording requirements for documentation in nonformulary notes were updated, and the high percentage of "no" ratings in the first and second quarter 2009 might indicate that confusion surrounded the changes in wording and documentation expectations. The increased number of "no" ratings found in these 2 quarters prompted additional efforts to educate PCCPS team members on the new requirements. Subsequent improvements in 2009 Q3 and Q4 could possibly be attributed to this group education.
It is more difficult to explain why there was not a consistent significant improvement in the documentation in the CDTM section. A high number of the errors in 2007 Q2 were due to lack of updating a physician referral for CDTM under the appropriate tab in the EMR. Many CPS would document the reason for referral in the subjective portion of the note but forget to document the referral in the "FYI" (for your information) tab in the EMR. The process for appropriate documentation was reviewed at a PCCPS monthly staff meeting, and subsequent documentation of this element did improve. This particular error now occurs very rarely and is mostly caused by new practitioners less familiar with the process.
Initially, the peer review process was met with some skepticism. Many PCCPS members were anxious regarding the process, and many questioned how the results would be used. Similarly, reports by Cram et al. and Haines et al. described concerns voiced early in the process by clinical practitioners regarding personal reaction to peer reviews and possible tensions resulting from constructive criticism. 5,6 Currently, CPS report little or no apprehension surrounding peer review. Instead, they express appreciation for the constructive feedback from the peer review process and report that they are able to implement aspects of well-written notes into their own practices. The positive response by CPS to the peer review process is consistent with the study by Haines et al., in which a high percentage of pharmacists surveyed expressed the view that the review process was valuable and should be continued. 6 To date, results of the KPCO peer review process have not been used for disciplinary action. Instead, the results have conducted. It might be useful in the future to formally evaluate these measures to further improve the process and address anonymously staff concerns and feedback.

■■ Conclusions
Over the past few years, the peer review process has evolved within PCCPS and emerged as a simple yet effective way to monitor trends, ensure compliance with existing regulatory standards, and allow for sharing best practices among practitioners. It has helped to improve consistency in note writing, while allowing each practitioner to maintain his or her individualized documentation style. Improvements in PCCPS documentation have been demonstrated over time, with the most recent results indicating a very high level of performance and almost all notes containing the necessary elements of documentation.